Interactive image marking method, electronic device, and recording medium using the method

ABSTRACT

An interactive image marking method is introduced. The interactive image marking method includes the following steps, displaying a target image and at least one marked region in the target image; receiving an interactive signal, where the interactive signal corresponds to a first pixel of the target image; calculating a correlation between the first pixel and pixels of the target image, and determining a correlation range in the target image according to the correlation; editing the marked region according to the correlate range; and displaying the edited marked region. In addition, an electronic device and a recording medium using the method are also introduced.

CROSS-REFERENCE TO RELATED APPLICATION

This non-provisional application claims priority under 35 U.S.C. §119(a) on Patent Application No. 110125657 filed in Taiwan, R.O.C. onJul. 13, 2021, the entire contents of which are hereby incorporated byreference.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present disclosure relates to software methods for computer-basedvision field, and in particular to an interactive image marking method.

2. Description of the Related Art

Diagnostic images are generated by modern medical imaging technologies,such as X-ray imaging, computed tomography, magnetic resonance imaging,nuclear medicine, and biopsies, to enable medical professionals to makeaccurate diagnoses. Therefore, some image analysis-oriented softwaretechnologies are applied to the reading and analyses of medical images.

Medical image analysis requirements vary from medical behavior orscenario to medical behavior or scenario. It is impractical to designspecific software dedicate to every type of medical behavior orscenario. Therefore, it is important to provide a system which meets theneeds of the largest possible number of users.

BRIEF SUMMARY OF THE INVENTION

An objective of the present disclosure is to provide an interactiveimage marking method, electronic device, recording medium and computerprogram product using the method, so as to enable interaction to takeplace between a user and a target image and determine how to mark thetarget image according to a signal sent by the user.

In an embodiment of the present disclosure, an interactive image markingmethod comprises the steps of: displaying a target image and at leastone marked region in the target image; receiving an interactive signal,wherein the interactive signal corresponds to a first pixel of thetarget image; calculating a first correlation between the first pixeland pixels of the target image; determining a correlation range in thetarget image according to the first correlation; editing the at leastone marked region according to the correlation range; and displaying theat least one marked region edited.

In an embodiment of the present disclosure, an electronic devicecomprises an input component, display unit, storage element andprocessor. The processor is coupled to the input component, display unitand storage element. The input component receives signals. The displayunit displays images. The storage element stores modules, including animage displaying module, signal receiving module, range determiningmodule and marking interaction module. The processor is adapted toaccess and execute the modules stored in the storage element. The imagedisplaying module displays a target image and at least one marked regionin the target image on the display unit. The signal receiving modulereceives an interactive signal through the input component. Theinteractive signal corresponds to a first pixel in the target image. Therange determining module calculates a first correlation between thefirst pixel and pixels of the target image and determines a correlationrange in the target image according to the first correlation. Themarking interaction module edits the at least one marked regionaccording to the correlation range. Moreover, the image displayingmodule further displays the marked region edited.

In an embodiment of the present disclosure, a non-transitorycomputer-readable recording medium for storing therein a program isprovided. The interactive image marking method is carried out as soon asthe program is loaded to a computer and executed thereon.

In an embodiment of the present disclosure, a computer program productstoring therein a computer program and adapted for use in interactiveimage marking. The interactive image marking method is carried out assoon as the program is loaded to a computer and executed thereon.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an electronic device in an embodiment ofthe present disclosure.

FIG. 2 is a schematic view of the process flow of an interactive imagemarking method according to an embodiment of the present disclosure.

FIG. 3 is a schematic view of a target image according to an embodimentof the present disclosure.

FIG. 4 is a schematic view of the interactive image marking methodaccording to an embodiment of the present disclosure.

DETAILED DESCRIPTION OF THE INVENTION

To facilitate understanding of the object, characteristics and effectsof this present disclosure, embodiments together with the attacheddrawings for the detailed description of the present disclosure areprovided.

Some of the embodiments of the present disclosure are hereunder depictedwith accompanying drawings and described below. Regarding referencenumerals used hereunder, when identical reference numerals are shown indifferent accompanying drawings, the reference numerals shall be deemeddenoting identical or similar components. These embodiments are merelyparts of the present disclosure but do not disclose all feasibleembodiments of the present disclosure. To be precise, these embodimentsmerely define the scope of a method, device and system disclosed in theclaims of the present disclosure.

The embodiments described hereunder are about computed tomography (CT)images. Each CT image is segmented into object regions, for example, apatient's internal organs, including the liver, kidneys, gallbladder,pancreas, spleen and lesions. From these object regions, users selectone or more object regions of interest. The selected object regions aremarked and displayed on a display unit and known as “marked region” inthis embodiment. This embodiment provides an interactive image markingmethod whereby users quickly, conveniently and precisely edit or alterthe scope of the marked region, for example, create marked regions,delete a part of a marked region or expand an existing marked region,but the present disclosure is not limited thereto. For instance, when aliver-related part of a computed tomography image is not completelysegmented into one single object region, users can mark only “part ofliver range” in the computed tomography image by selecting the objectregion. By contrast, the interactive image marking method in thisembodiment allows a marked “part of liver range” to be expanded quickly,conveniently and precisely to become “full liver range”. However, thepresent disclosure is not limited thereto. The interactive image markingmethod of the present disclosure is also applicable to images of anyother scenarios and fields.

FIG. 1 is a block diagram of an electronic device in an embodiment ofthe present disclosure.

Referring to FIG. 1 , in this embodiment, an electronic device 100comprises a display unit 110, storage element 120, processor 130 andinput component 140. The processor 130 is coupled to the display unit110, storage element 120 and input component 140. The electronic device100 is capable of performing image computation and is, for example, apersonal computer, notebook computer, smartphone, tablet or personaldigital assistant (PDA), but is not limited thereto.

The display unit 110 displays images and provides the images to usersfor watching. In this embodiment, display unit 110 is, for example, aliquid crystal display (LCD), light-emitting diode (LED) or fieldemission display (FED), but the present disclosure is not limitedthereto.

The storage element 120 is, for example, fixed or movable, is randomaccess memory (RAM), read-only memory (ROM), Flash memory, hard diskdrive, the like, or a combination thereof, and is adapted to store filesand data. In this embodiment, the storage element 120 records modulesexecuted by the processor 130. The modules include an image displayingmodule 121, signal receiving module 123, range determining module 125and marking interaction module 127.

The processor 130 is, for example, a central processing unit (CPU),microprocessor which is programmable to serve general or specialpurposes, digital signal processor (DSP), programmable controller,Application Specific Integrated Circuits (ASIC), programmable logicdevice (PLD), the like or a combination thereof. In this embodiment, theprocessor 130 is coupled to the display unit 110, storage element 120and input component 140 to access the modules in the storage element 120and thus work together with the display unit 110 and input component 140to effect interactive image marking.

The input component 140 enables users to operate the electronic device100 and receives signals generated as a result of user operation. Theinput component 140 is a keyboard, mouse, stylus, touch panel ortrackball, which is externally connected to or built-in in theelectronic device 100 and is coupled to the processor 130. In someembodiments, the input component 140 is integrated into the display unit110 to form a touchscreen, such as a capacitive or resistive touchscreenfor receiving users' touch-based inputs.

FIG. 2 is a schematic view of the process flow of an interactive imagemarking method according to an embodiment of the present disclosure. Themethod applies to the electronic device 100 of FIG. 1 . The details ofthe interactive image marking method of the present disclosure arehereunder described with reference to the constituent elements of theelectronic device 100, but the interactive image marking method is notrestricted to the electronic device 100.

Referring to FIG. 2 , in step S201, the image displaying module 121displays the target image and the current marked region in the targetimage on the display unit 110.

The target image comprises computer images of multiple pixels. Themarked region is the marked region in the target image. The marks inregions vary from marked region to marked region. In some embodiments,after the display unit 110 has shown a target image, users use the inputcomponent 140 to interact with the displayed contents, for example,marking one or more specific regions in the target image to allow themto become marked regions, and the display unit 110 displays the currentinteraction result, for example, a marked region of the target image,but the present disclosure is not limited thereto.

The present disclosure is not restrictive of the way of marking ordifferentiating marked regions. In some embodiments, marked regions aremarked and differentiated by color, allowing identical marked regions tobe marked in the same color, and different marked regions to be markedin different colors. In some embodiments, marked regions are marked anddifferentiated through annotation.

In this embodiment, step 201 further comprises step S2011 and stepS2013.

Referring to FIG. 2 , in step S2011, the image segmentation module 121of the electronic device 100 executes an image segmentation algorithm onthe target image to obtain a result of image segmentation.

The image segmentation algorithm performs image segmentation on thetarget image and is, for example, a foreground segmentation algorithmfor use in computer vision field. The image segmentation algorithm, forexample, performs preprocessing, recognition, sorting and the like onthe target image to segment the target image into multiple objectregions. Take vehicular automation as an example, the target image is,for example, an image taken with a vehicular panoramic camera, and theimage segmentation algorithm, for example, segments the image intomultiple object regions related to a road, pedestrian, traffic light,street tree, vehicle, respectively. Take medical field as an example,the target image is, for example, a computed tomography image, and theimage segmentation algorithm, for example, segments the image intoobject regions related to the liver, kidneys, gallbladder, pancreas,spleen and lesions, respectively, to function as an image segmentationresult. However, the present disclosure is not limited thereto, andpersons skilled in the art can design an image segmentation algorithmaccording to the target images, computation capabilities, and needsdisclosed herein.

FIG. 3 is a schematic view of a target image and a marked region in thetarget image according to an embodiment of the present disclosure.

Referring to FIG. 3 , in this embodiment, target image TI is, forexample, a computed tomography image, and the image segmentationalgorithm is, for example, Graph Based Segmentation Algorithm in OpenCV,but the present disclosure is not limited thereto. In this embodiment,the image segmentation algorithm segments target image TI into liver OR1marked in a grill-like pattern, kidneys OR2 marked in a lattice-likepattern, and object regions (not shown) related to gallbladder,pancreas, and spleen. In this embodiment, the image segmentationresults, for example, comprises liver OR1, kidneys OR2, and objectregions (not shown) related to gallbladder, pancreas, and spleen.

In step S2013, the image displaying module 121 displays a target imageand a current marked region in the target image on the display unit 110.In step S203, the signal receiving module 123 receives an interactivesignal through the input component 140.

In this embodiment, the interaction taking place between a target imageand a user through the input component 140 comprises “increase”,“expansion” and “deletion”. Thus, the interactive signals include anincrease signal, deletion signal and expansion signal. The “increase”interaction involves creating a new marked region in the target image.The “expansion” interaction involves expanding a specific marked regionin the target image to expand the marking range of the marked region.The “delete” interaction involves deleting at least one portion of allthe marked regions in the target image. The present disclosure is notrestrictive of the switching method of the aforesaid three interactionmodes, and thus persons skilled in the art can design related hardwareor software as needed.

In this embodiment, if the target image has not yet got any markings(that is, when the target image has not yet got any marked regionstherein,) the display unit 110 will, in step S2013, for example, displaya target image which has no marked regions. Conversely, if the targetimage has a marked region therein, the display unit 110 will, in stepS2013, display a target image and current marked regions in the targetimage (that is, simultaneously display a target image and marked regionswith marks.)

Referring to FIG. 3 , in this embodiment, users, for example, use theinput component 140 to mark two object regions corresponding in positionto liver and kidneys, to create two marked regions OR1, OR2. Therefore,the display unit 110 displays target image TI and marks the markedregion OR1 (i.e., liver) and marked region OR2 (i.e., kidneys) in targetimage TI. The details about users' using the input component 140 to markthe object regions as marked regions are discussed below.

In step S205, the processor 130 determines whether an interactive signalreceived by the signal receiving module 123 is an increase signal ADD,expansion signal ENL or deletion signal DEL.

In some embodiments, users use the input component 140 to switch theelectronic device 100 to the “increase” interaction mode and then sendthe increase signal ADD corresponding to a region to be marked, forexample, by applying a first-category stroke to the region to be marked.Thus, the increase signal ADD corresponds to one or more pixels.

In some embodiments, users use the input component 140 to switch theelectronic device 100 to “expansion” interaction mode and send theexpansion signal ENL corresponding to an expansion region, for example,by applying a first-category stroke to the region to be expanded. Thus,the expansion signal ENL corresponds to one or more pixels.

In some embodiments, users use the input component 140 to switch theelectronic device 100 to “delete” interaction mode and then send thedeletion signal DEL corresponds to a region to undergo mark deletion,for example, by applying a first-category stroke to a region to undergomark deletion. Therefore, the deletion signal DEL corresponds to one ormore pixels.

The first-category stroke is different from a second-category stroke(The second-category stroke is defined and described later.) Forinstance, the first-category stroke is in a specific color and thesecond-category stroke in another. For instance, the first-categorystroke is applied with the left key of a mouse and the second-categorystroke with the right key of the mouse. However, the present disclosureis not limited thereto.

If, in step S205, the processor 130 determines that the interactivesignal received by the signal receiving module 123 is the increasesignal ADD, the process flow of the method goes to step S207, allowingthe range determining module 125 to determine the correlation rangecorresponding to the increase signal ADD. Then, the process flow of themethod goes to step S209, allowing the marking interaction module 127 tocreate a marked region according to a correlation range.

In step S207, the range determining module 125 or marking interactionmodule 127 determines the pixels corresponding to the increase signalADD, and then the range determining module 125 calculates thecorrelation range corresponding to the increase signal ADD according tothe pixels corresponding to the increase signal ADD.

FIG. 4 is a schematic view of the interactive image marking methodaccording to an embodiment of the present disclosure.

Referring to FIG. 4 , in this embodiment, marked regions OR1, OR2 are,for example, two object regions corresponding in position to the liverand kidneys, respectively, and the increase signal ADD is, for example,first-category stroke S1. The range determining module 125 or markinginteraction module 127 selects a predetermined number (for example,five) of equidistantly spaced apart pixels according to first-categorystroke S1 to function as the pixels corresponding to the increase signalADD. The range determining module 125 calculates first correlationbetween all the pixel points of unmarked regions in target image TI andthe pixels corresponding to the increase signal ADD.

One or more features in the same marked region have high similaritylevels. Therefore, the range determining module 125 calculatessimilarity levels of one or more features between the pixelscorresponding to the increase signal ADD and all the pixel points ofunmarked regions in target image TI. In some embodiments, the aforesaidfeatures are, for example, color, texture, size or shape, but thepresent disclosure is not limited thereto.

In this embodiment, the range determining module 125 uses, for example,a machine learning model, to calculate the first correlation between thepixels corresponding to the increase signal ADD and the pixel points ofunmarked regions in target image TI. For instance, the range determiningmodule 125 calculates the average of the values of similarity levelsbetween first pixel points of unmarked regions in target image TI andpixels corresponding to the increase signal ADD, respectively, andallows the calculated average to function as a first correlation betweenfirst pixel points and pixels corresponding to the increase signal ADD.Furthermore, the range determining module 125 calculates the average ofthe values of similarity levels between second pixel points of unmarkedregions in target image TI and pixels corresponding to the increasesignal ADD, respectively, and allows the calculated average to functionas a first correlation between second pixel points and pixelscorresponding to the increase signal ADD.

In this embodiment, the machine learning model, for example, consists ofthree modules, namely Distance maps fusion module, Pre-trained ResNetbackbone and DeepLabV3+Decoder, whereby similarity levels between pixelpoints of unmarked regions in target image TI and pixels correspondingto the increase signal ADD are outputted to function as the firstcorrelation. However, the present disclosure is not restrictive of themachine learning model and will work provided that similarity levelsbetween pixel points in target image TI and pixels corresponding to theincrease signal ADD are calculated; thus, persons skilled in the art candesign the present disclosure as needed.

In this embodiment, the range determining module 125 presets a firstcorrelation threshold, selects, from all the pixel points of unmarkedregions in target image TI, the pixel points whose first correlation ishigher than the first correlation threshold, and allows the selectedpixel points to function as correlation range CR (depicted as a whiteregion which includes the first-category stroke S1) corresponding to theincrease signal ADD.

In step S209, the marking interaction module 127 marks the pixels incorrelation range CR corresponding to the increase signal ADD and allowsthe marked pixels to function as a new marked region. Finally, theprocess flow of the method returns to step S2013, allowing the displayunit 110 to display target image TI and current marked regions. The newmarked region is different from the current marked region in terms ofmarks. For instance, the current marked region comprises liver OR1marked in a grill-like pattern and kidneys OR2 marked in a lattice-likepattern, whereas a new marked region is marked in a pattern which isneither grill-like nor lattice-like. For instance, the current markedregion comprises liver OR1 marked in red and kidneys OR2 marked in blue,whereas a new marked region is marked neither in red nor in blue (forexample, is marked in green.) However, the present disclosure is notlimited thereto.

In some embodiments, users, for example, the first correlation thresholdis adjusted with the input component 140 to adjust the aforesaidcorrelation range CR and thus adjust the range of the new marked region.

The input component 140 not only sends interactive signals but alsosends inhibition interactive signals, so as to circumvent the inhibitionrange corresponding to the inhibition interactive signals (and thus notto edit the inhibition range) while editing a marked region according tothe correlation range corresponding to the interactive signals.

In some embodiments, in the “increase” interaction mode, users use theinput component 140 to not only send the increase signal ADD (forexample, the first-category stroke S1) corresponding to a region to bemarked but also send the inhibition increase signal corresponding to an“unmarked region” to prevent the “unmarked region” from being markedbecause of the increase signal ADD. The inhibition increase signal is,for example, to apply a second-category stroke S2 in an unmarked region.Therefore, like the increase signal ADD, the inhibition increase signalalso corresponds to one or more pixels.

Likewise, the range determining module 125 calculates all the pixelpoints of unmarked regions in target image TI and the second correlationbetween the pixels corresponding to the inhibition increase signal.Then, the range determining module 125 sets a second correlationthreshold, selects, from all the pixel points of unmarked regions intarget image TI, the pixel points whose second correlation is higherthan the second correlation threshold, and let them define inhibitionrange IR (depicted as a white region which includes the second-categorystroke S2.) When correlation range CR is marked with the interactionmodule 127 to create a new marked region, inhibition range IR is notmarked. Thus, even if the correlation between the pixel points ininhibition range IR and the pixels corresponding to the increase signalADD is high, the pixel points in inhibition range IR will not be marked.

If, in step S205, the processor 130 determines that the interactivesignal received by the signal receiving module 123 is the deletionsignal DEL, the process flow of the method goes to step S211, allowingthe range determining module 125 to determine the correlation rangecorresponding to the deletion signal DEL. Then, the process flow of themethod goes to step S213, in which the marking interaction module 127deletes marks in the correlation range from the marked region of targetimage TI.

Step S211 (the way of determining the correlation range corresponding tothe deletion signal DEL) is similar to step S207 (the way of determiningthe correlation range corresponding to the increase signal ADD). In thisembodiment, the range determining module 125 calculates the firstcorrelation between the pixel points of all the marked regions in targetimage TI and the pixels corresponding to the deletion signal DEL,respectively. Then, the range determining module 125 sets the firstcorrelation threshold, selects, from all the pixel points of all themarked regions in target image TI, the pixel points whose firstcorrelation is higher than the first correlation threshold, and allowsthe selected pixel points to function as a correlation range (not shown)corresponding to the deletion signal DEL. The correlation range CRcorresponding to the increase signal ADD lies in unmarked regions oftarget image TI, and the correlation range corresponding to the deletionsignal DEL lies in marked regions of target image TI.

In step S213, the marking interaction module 127 deletes marks of pixelpoints in the correlation range from target image TI. Finally, theprocess flow of the method returns to step S2013, allowing the displayunit 110 to display target image TI and current marked regions.

In some embodiments, users, for example, employ the input component 140to adjust the first correlation threshold for use in determining thecorrelation range corresponding to the deletion signal DEL to thereforefurther adjust the range of deleting marks.

In some embodiments, in the “delete” interaction mode, the inputcomponent 140 not only sends the deletion signal DEL corresponding toregions to be deleted but also sends the inhibition deletion signalcorresponding to regions not to be deleted, so as to prevent the regionsnot to be deleted from being deleted because of the deletion signal DEL.The inhibition deletion signal is, for example, to apply asecond-category stroke in the regions not to be deleted. Therefore, likethe deletion signal DEL, the inhibition deletion signal also correspondsto pixels.

Likewise, the range determining module 125 calculates the secondcorrelation between all the pixel points of marked regions in targetimage TI and pixels corresponding to the inhibition deletion signal.Then, the range determining module 125 sets a second correlationthreshold, selects, from all the pixel points of marked regions intarget image TI, the pixel points whose second correlation is higherthan the second correlation threshold, and allows the selected pixelpoints to function as the inhibition range. If the marking interactionmodule 127 is able to delete marks of pixel points in the correlationrange, marks of pixel points in the inhibition range will not bedeleted. Therefore, even if the correlation between the pixel points ininhibition range IR and the pixels corresponding to the deletion signalDEL is high, marks of the pixel points in the inhibition range IR cannotbe deleted.

If, in step S205, the processor 130 determines that the interactivesignal received by the signal receiving module 123 is the expansionsignal ENL, the marking interaction module 127 will determine the targetregion according to the expansion signal ENL.

In step S215, the marking interaction module 127 determines whether thetarget region is present according to the expansion signal ENL. If themarking interaction module 127 determines that no target regions exist,the process flow of the method goes to step S217, allowing the imagedisplaying module 121 to display a message of expansion failure on thedisplay unit 110. If the marking interaction module 127 determines thata target region is present, the process flow of the method goes to stepS219.

The marking interaction module 127 determines which of the markedregions in target image TI is to be expanded by the expansion signalENL. The marked region to be expanded by the expansion signal ENL is thetarget region. In some embodiments, the marking interaction module 127does not find any target regions corresponding to the expansion signalENL in target image TI.

Referring to FIG. 4 , in this embodiment, marked regions OR1, OR2 are,for example, two object regions corresponding in position to liver andkidneys, respectively. The expansion signal ENL is, for example,first-category stroke S1. The range determining module 125 or markinginteraction module 127 selects a predetermined number (for example,five) of pixels spaced apart equidistantly and corresponding in positionto the first-category stroke S1 and allows the selected pixels tofunction as the pixels corresponding to the expansion signal ENL.

The marking interaction module 127 calculates a distance relationbetween the pixels corresponding to the expansion signal ENL and markedregions OR1, OR2 and then selects a target region according to thedistance relation. For instance, the expansion signal ENL expands themarked region least far away from the expansion signal ENL. Therefore,the marked region least far away from the expansion signal ENL isselected to be the target region. However, the present disclosure is notlimited thereto.

In this embodiment, the marking interaction module 127 selects themarked region which is least far away from the expansion signal ENL andlies within the predetermined distance range of the pixels correspondingto the expansion signal ENL and allows the selected marked region to bethe target region. Upon determination that marked regions do not existwithin the distance range of the pixels corresponding to the expansionsignal ENL, the process flow of the method goes to step S217.Conversely, if at least one marked region is present within thepredetermined distance range of the pixels corresponding to theexpansion signal ENL, the marking interaction module 127 selects themarked region least far away from the pixels corresponding to theexpansion signal ENL and allows the selected marked region to functionas a target region.

For instance, the pixels corresponding to the expansion signal ENLcomprise a first pixel, second pixel, third pixel, fourth pixel andfifth pixel. The marking interaction module 127 searches a square block(centered at the first pixel and having a length of 10 pixels, forexample) for the marked region least far away from the first pixel andrecords both the marked region and the distance between the selectedmarked region and the first pixel. Furthermore, the marking interactionmodule 127 searches a square block (centered at the second pixel andhaving a length of 10 pixels, for example) for the marked region leastfar away from the second pixel and records both the marked region andthe distance between the selected marked region and the second pixel. Byanalogy, the marking interaction module 127 deals with the third pixel,fourth pixel and fifth pixel as well. A first marking block exists in asquare block centered at the first pixel and is separated from the firstpixel by the least distance of eight pixels. No marking block is foundin square blocks centered at second pixel and third pixel. A firstmarking block and a second marking block exist in a square blockcentered at the fourth pixel and are separated from the fourth pixel bythe shortest distance of eight pixels and 12 pixels, respectively. Afirst marking block exists in a square block centered at the fifth pixeland is separated from the fifth pixel by the shortest distance of sixpixels. The marking interaction module 127 determines that the markingblock nearest to the pixels corresponding to the expansion signal ENL isthe first marking block and allows it to function as a target region. Itis because the first marking block is separated from the pixelscorresponding to the expansion signal ENL by the shortest distance ofsix pixels.

In this embodiment, the marking interaction module 127, for example,determines that the target region is marked region OR1, and the processflow of the method goes to step S219.

Like step S207, in step S219, the range determining module 125determines the pixels corresponding to the expansion signal ENL and thencalculates the correlation range corresponding to the expansion signalENL according to the pixels corresponding to the expansion signal ENL.

In this embodiment, the range determining module 125 calculates a firstcorrelation between each pixel point of an unmarked region in targetimage TI and the pixels corresponding to the expansion signal ENL. Then,the range determining module 125 sets a first correlation threshold,searches all the pixel points of unmarked regions in target image TI forthe pixel points whose first correlation is higher than the firstcorrelation threshold, and allows them to function as correlation rangeCR corresponding to the expansion signal ENL. Thus, correlation range CRcorresponding to the expansion signal ENL lies in the unmarked regionsof target image TI. In this embodiment, the first correlation is, forexample, the similarity level of one or more features, but the presentdisclosure is not limited thereto.

In some embodiments, the range determining module 125 sets a firstcorrelation threshold according to the distance between each pixelcorresponding to the expansion signal ENL and a target region, whereasthe distance between the first correlation threshold, the pixelscorresponding to the expansion signal ENL, and the target region arenegatively correlated. Thus, the greater the distance between the pixelscorresponding to the expansion signal ENL and the target region is, theless is the first correlation threshold, and the greater is thecorrelation range corresponding to the expansion signal ENL.

For instance, the range determining module 125 calculates a firstcorrelation threshold according to the equation below.

TH1=1−C*(dis)^(1/2)

TH1 denotes the first correlation threshold, dis denotes the distancebetween each pixel corresponding to the expansion signal ENL and atarget region, and C is a constant. In some embodiments, C is, forexample, 0.02, but the present disclosure is not limited thereto.

Moreover, in some embodiments, the distance dis between each pixelcorresponding to the expansion signal ENL and a target region is theleast distance between each pixel corresponding to the expansion signalENL and a target region in step S215, for example, six pixels. In someembodiments, the distance dis between each pixel corresponding to theexpansion signal ENL and a target region is also the least distancebetween the center of the first-category stroke S1 and the targetregion. However, the present disclosure is not restricted to theaforesaid definition of the distance dis, and persons skilled in the artcan design its definition as needed.

Referring to FIG. 2 , in step S221, the processor 130 determines whetherthe signal receiving module 123 has received an inhibition expansionsignal. If the signal receiving module 123 has not received anyinhibition expansion signal, the process flow of the method goes to stepS223, allowing the marking interaction module 127 to expand the targetregion according to the correlation range.

In step S223, the marking interaction module 127 expands the markingcorrelation range to a target region, by adding marks to the pixels inthe correlation range, wherein the marks added are the same as those inthe target region. After that, the process flow of the method returns tostep S2013, displaying target image TI and current marked region on thedisplay unit 110. For instance, the current marked region, for example,comprises marked region OR1 (i.e., liver) marked in a grill-like patternand marked region OR2 (i.e., kidneys) marked in a lattice-like pattern,wherein the target region is, for example, marked region OR1 marked in agrill-like pattern. The marking interaction module 127 adds grill-likemarks to the pixel points in the correlation range to expand targetregion OR1 marked in a grill-like pattern. For instance, the currentmarked region comprises marked region OR1 (i.e., liver) marked in redand marked region OR2 (i.e., kidneys) marked in blue, wherein the targetregion is, for example, marked region OR1 marked in red. The markinginteraction module 127 adds red marks to the pixel points in thecorrelation range to expand target region OR1 marked in red.

In some embodiments, users, for example, adjust the first correlationthreshold with the input component 140 to adjust the correlation rangeand thus further adjust the expansion range of a target region.

If the signal receiving module 123 receives an inhibition expansionsignal, the process flow of the method goes to step S225. In step S225,the range determining module 125 or marking interaction module 127determines pixels corresponding to an inhibition expansion signal, andthen the range determining module 125 calculates the inhibition rangecorresponding to the inhibition expansion signal according to the pixelscorresponding to the inhibition expansion signal.

In this embodiment, in the “expansion” interaction mode, users employthe input component 140 to not only send the expansion signal ENLcorresponding to regions to be expanded but also send an inhibitionexpansion signal corresponding to the “non-expansion regions” to preventthe “non-expansion regions” from being marked because of the expansionsignal ENL. The inhibition expansion signal is, for example, to applythe second-category stroke S2 in the “non-expansion regions”. Therefore,like the expansion signal ENL, the inhibition expansion signalcorresponds to one or more pixels.

Referring to FIG. 4 , in this embodiment, marked regions OR1, OR2 are,for example, two object regions related to liver and kidneys,respectively. The expansion signal ENL is, for example, thefirst-category stroke S1. The inhibition expansion signal is, forexample, the second-category stroke S2. The range determining module 125selects a predetermined number (for example, five) of equidistantlyspaced apart pixel points according to the second-category stroke S2 andallows the selected pixel points to function as the pixels correspondingto the inhibition expansion signal.

Likewise, the range determining module 125 calculates a secondcorrelation between all the pixel points of unmarked regions in thetarget image TI and pixels corresponding to the inhibition increasesignal. Then, the range determining module 125 sets a second correlationthreshold, searches all the pixel points of unmarked regions in targetimage TI for the pixel points whose second correlation is higher thanthe second correlation threshold, and allows the pixel points tofunction as inhibition range IR. In this embodiment, the secondcorrelation is, for example, a similarity level of one or more features,but the present disclosure is not limited thereto.

In step S227, the marking interaction module 127 expands the targetregion according to the correlation range and inhibition range.

In step S227, the marking interaction module 127 expands the markingcorrelation range to the target region but does not expand the markinginhibition range to the target region. Finally, the process flow of themethod returns to step S2013 in which the display unit 110 displaystarget image TI and a current marked region. For instance, the currentmarked region, for example, comprises liver OR1 marked in red andkidneys OR2 marked in blue, wherein the current marked region, forexample, comprises the target region (liver OR1) marked in red. Themarking interaction module 127 adds red marks to the pixel points incorrelation range CR, so as to expand marking correlation range CR tothe target region OR1 while avoiding expanding marks to pixel points ininhibition range IR. Thus, even if the correlation between the pixelpoints in inhibition range IR and the pixels corresponding to theexpansion signal ENL is high, the pixel points will not be expanded andmarked to the target region.

An embodiment of the present disclosure provides a computer-readablerecording medium for storing therein a program, as exemplified by thestorage element 120 shown in FIG. 1 . The interactive image markingmethod described in the aforesaid embodiments is carried out as soon asthe program is loaded to a computer and executed thereon.

An embodiment of the present disclosure provides a computer programproduct storing therein a computer program and adapted for use ininteractive image marking. The interactive image marking methoddescribed in the aforesaid embodiments is carried out as soon as theprogram is loaded to a computer and executed thereon. Therefore, theprogram or software for use in the interactive image method is stored inthe computer-readable recording medium. Furthermore, the computerprogram product is provided through, for example, network-basedtransmission.

The embodiments of the present disclosure provide an interactive imagemarking method, electronic device, recording medium and computer programproduct using the method to allow users to intuitively edit markedregions in a target image with interactive signals, including creating amarked region, deleting part of a mark, and expanding a marked region.Moreover, for the sake of editing, the interactive image marking methodis effective in precisely determining the correlation rangecorresponding to an interactive signal according to an interactivesignal from a user. Thus, the present disclosure is conducive toenhancement of marking efficiency and precision of target images and theresultant user experience.

While the present disclosure has been described by means of specificembodiments, numerous modifications and variations could be made theretoby those skilled in the art without departing from the scope and spiritof the present disclosure set forth in the claims.

What is claimed is:
 1. An interactive image marking method, comprisingthe steps of: displaying a target image and at least one marked regionin the target image; receiving an interactive signal, wherein theinteractive signal corresponds to a first pixel of the target image;calculating a first correlation between the first pixel and pixels ofthe target image; determining a correlation range in the target imageaccording to the first correlation; editing the at least one markedregion according to the correlation range; and displaying the at leastone marked region edited.
 2. The interactive image marking method ofclaim 1, wherein the interactive signal is a first-category stroke. 3.The interactive image marking method of claim 2, wherein the step ofcalculating the first correlation between the first pixel and the pixelsof the target image comprises selecting a predetermined number of pixelsspaced apart equidistantly and corresponding in position to thefirst-category stroke and allowing the selected pixels to function asthe first pixel.
 4. The interactive image marking method of claim 1,further comprising: receiving an inhibition interactive signal, whereinthe inhibition interactive signal corresponds to a second pixel in thetarget image; calculating a second correlation between the second pixeland pixels of the target image; and determining an inhibition range inthe target image according to the second correlation, wherein editingthe at least one marked region according to the correlation range is notaccompanied by editing the inhibition range.
 5. The interactive imagemarking method of claim 4, wherein the interactive signal is afirst-category stroke, and the inhibition interactive signal is asecond-category stroke.
 6. The interactive image marking method of claim1, wherein the interactive signal is an increase signal, and the step ofediting the at least one marked region according to the correlationrange comprises marking the correlation range so as to create the atleast one marked region.
 7. The interactive image marking method ofclaim 6, further comprising: receiving an inhibition increase signal,wherein the inhibition increase signal corresponds to a second pixel inthe target image; calculating a second correlation between the secondpixel and pixels of the target image; and determining an inhibitionrange in the target image according to the second correlation, whereinmarking the correlation range to create the at least one marked regionis not accompanied by marking the inhibition range.
 8. The interactiveimage marking method of claim 7, wherein the increase signal is afirst-category stroke, and the inhibition increase signal is asecond-category stroke.
 9. The interactive image marking method of claim1, wherein the interactive signal is a deletion signal, and the step ofediting the at least one marked region according to the correlationrange comprises deleting marks in the correlation range from the atleast one marked region.
 10. The interactive image marking method ofclaim 9, further comprising: receiving an inhibition deletion signal,wherein the inhibition deletion signal corresponds to a second pixel inthe target image; calculating a second correlation between the secondpixel and pixels of the target image; and determining an inhibitionrange in the target image according to the second correlation, whereindeleting marks in the correlation range from the at least one markedregion is not accompanied by deleting marks in the inhibition range. 11.The interactive image marking method of claim 10, wherein the deletionsignal is a first-category stroke, and the inhibition deletion signal isa second-category stroke.
 12. The interactive image marking method ofclaim 1, wherein the interactive signal is an expansion signal, and theinteractive image marking method further comprises: calculating adistance relation between the first pixel and the at least one markedregion; and selecting a target region from the at least one markedregion according to the distance relation, wherein the step of editingthe at least one marked region according to the correlation rangecomprises marking the correlation range to expand the target region. 13.The interactive image marking method of claim 12, wherein the step ofcalculating the first correlation between the first pixel and the pixelsof the target image comprises: determining a similarity thresholdaccording to a distance between the first pixel and the target region,wherein the similarity threshold and the distance are negativelycorrelated; and calculating first similarity levels between the firstpixel and the pixels of the target image and allowing the firstsimilarity levels to function as the first correlation.
 14. Theinteractive image marking method of claim 13, wherein the step ofdetermining the correlation range in the target image according to thefirst correlation comprises determining the correlation range accordingto the first similarity levels and the similarity level threshold. 15.The interactive image marking method of claim 14, further comprising:receiving an inhibition expansion signal, wherein the inhibitionexpansion signal corresponds to a second pixel in the target image;calculating second similarity levels between the second pixel and pixelsin the target image; and determining an inhibition range according tothe second similarity levels, wherein marking the correlation range toexpand the target region is not accompanied by marking the inhibitionrange.
 16. The interactive image marking method of claim 15, wherein theexpansion signal is a first-category stroke, and the inhibitionexpansion signal is a second-category stroke.
 17. The interactive imagemarking method of claim 5, wherein the first-category stroke and thesecond-category stroke differ in color.
 18. A non-transitorycomputer-readable recording medium for storing therein a program,wherein the interactive image marking method of claim 1 is carried outas soon as the program is loaded to a computer and executed thereon. 19.An electronic device, comprising: an input component for receiving asignal; a display unit for displaying an image; a storage element forstoring modules; and a processor coupled to the input component, thedisplay unit and the storage element and adapted to access and executethe storage element for storing the modules, the modules comprising: animage displaying module for displaying a target image and at least onemarked region in the target image on the display unit; a signalreceiving module for receiving an interactive signal through the inputcomponent, wherein the interactive signal corresponds to a first pixelin the target image; a range determining module for calculating a firstcorrelation between the first pixel and pixels of the target image anddetermining a correlation range in the target image according to thefirst correlation; and a marking interaction module for editing the atleast one marked region according to the correlation range, wherein theimage displaying module displays the at least one marked region edited.20. The electronic device of claim 19, wherein the interactive signal isa first-category stroke.
 21. The electronic device of claim 20, whereincalculating the first correlation between the first pixel and the pixelsof the target image with the range determining module is furtheraccompanied by selecting a predetermined number of pixels spaced apartequidistantly and corresponding in position to the first-category strokeand allowing the selected pixels to function as the first pixel.
 22. Theelectronic device of claim 19, wherein the signal receiving modulereceives an inhibition interactive signal through the input component,whereas the inhibition interactive signal corresponds to a second pixelin the target image, wherein the range determining module: calculates asecond correlation between the second pixel and pixels of the targetimage; and determines an inhibition range in the target image accordingto the second correlation, wherein editing the at least one markedregion according to the correlation range is not accompanied by editingthe inhibition range.
 23. The electronic device of claim 22, wherein theinteractive signal is a first-category stroke, and the inhibitioninteractive signal is a second-category stroke.
 24. The electronicdevice of claim 19, wherein the interactive signal is an increasesignal, wherein the marking interaction module's editing the at leastone marked region according to the correlation range is accompanied bythe marking interaction module's marking the correlation range to createthe at least one marked region.
 25. The electronic device of claim 24,wherein the signal receiving module receives an inhibition increasesignal through the input component, and the inhibition increase signalcorresponds to a second pixel in the target image, wherein the rangedetermining module: calculates a second correlation between the secondpixel and pixels of the target image; and determines an inhibition rangein the target image according to the second correlation, wherein markingthe correlation range to create the at least one marked region is notaccompanied by marking the inhibition range.
 26. The electronic deviceof claim 25, wherein the increase signal is a first-category stroke, andthe inhibition increase signal is a second-category stroke.
 27. Theelectronic device of claim 19, wherein the interactive signal is adeletion signal, wherein the marking interaction module's editing the atleast one marked region according to the correlation range isaccompanied by deleting marks in the correlation range from the at leastone marked region.
 28. The electronic device of claim 27, wherein thesignal receiving module receives an inhibition deletion signal throughthe input component, and the inhibition deletion signal corresponds to asecond pixel in the target image, wherein the range determining module:calculates a second correlation between the second pixel and pixels ofthe target image; and determines an inhibition range in the target imageaccording to the second correlation, wherein deleting marks in thecorrelation range from the at least one marked region is not accompaniedby deleting marks in the inhibition range.
 29. The electronic device ofclaim 28, wherein the deletion signal is a first-category stroke, andthe inhibition deletion signal is a second-category stroke.
 30. Theelectronic device of claim 19, wherein the interactive signal is anexpansion signal, wherein the marking interaction module: calculates adistance relation between the first pixel and the at least one markedregion; and selects the target region from the at least one markedregion according to the distance relation, wherein the markinginteraction module's editing the at least one marked region according tothe correlation range entails marking the correlation range to expandthe target region.
 31. The electronic device of claim 30, wherein therange determining module's calculating the first correlation between thefirst pixel and the pixels of the target image entails: determining asimilarity threshold according to a distance between the first pixel andthe target region, wherein the similarity threshold and the distance arenegatively correlated; and calculating first similarity levels betweenthe first pixel and the pixels of the target image and allowing thefirst similarity levels to function as the first correlation.
 32. Theelectronic device of claim 31, wherein the range determining module'sdetermining the correlation range in the target image according to thefirst correlation entails determining the correlation range according tothe first similarity levels and the similarity level threshold.
 33. Theelectronic device of claim 32, wherein the signal receiving modulereceives an inhibition expansion signal through the input component, andthe inhibition expansion signal corresponds to a second pixel in thetarget image, wherein the range determining module: calculates secondsimilarity levels between the second pixel and pixels in the targetimage; and determines an inhibition range according to the secondsimilarity levels, wherein marking the correlation range to expand thetarget region is not accompanied by marking the inhibition range. 34.The electronic device of claim 33, wherein the expansion signal is afirst-category stroke, and the inhibition expansion signal is asecond-category stroke.
 35. The electronic device of claim 23, whereinthe first-category stroke and the second-category stroke differ incolor.